PILOT: Efficient Planning by Imitation Learning and Optimisation for Safe Autonomous Driving

Achieving the right balance between planning quality, safety and efficiency is a major challenge for autonomous driving. Optimisation-based motion planners are capable of producing safe, smooth and comfortable plans, but often at the cost of runtime efficiency. On the other hand, naively deploying trajectories produced by efficient-to-run deep imitation learning approaches might risk compromising safety. In this paper, we present PILOT -- a planning framework that comprises an imitation neural network followed by an efficient optimiser that actively rectifies the network's plan, guaranteeing fulfilment of safety and comfort requirements. The objective of the efficient optimiser is the same as the objective of an expensive-to-run optimisation-based planning system that the neural network is trained offline to imitate. This efficient optimiser provides a key layer of online protection from learning failures or deficiency on out-of-distribution situations that might compromise safety or comfort. Using a state-of-the-art, runtime-intensive optimisation-based method as the expert, we demonstrate in simulated autonomous driving experiments in CARLA that PILOT achieves a significant reduction in runtime when compared to the expert it imitates without sacrificing planning quality.Comment: 8 pages, 7 figure

Knowledge Capture in CMM Inspection Planning: Barriers and Challenges

Coordinate Measuring Machines (CMM) have been widely used as a means of evaluating product quality and controlling quality manufacturing processes. Many techniques have been developed to facilitate the generation of CMM measurement plans. However, there are major gaps in the understanding of planning such strategies. This significant lack of explicitly available knowledge on how experts prepare plans and carry out measurements slows down the planning process, leading to the repetitive reinvention of new plans while preventing the automation or even semi-automation of the process. The objectives of this paper are twofold: (i) to provide a review of the existing inspection planning systems and discuss the barriers and challenges, especially from the aspect of knowledge capture and formalization; and (ii) to propose and demonstrate a novel digital engineering mixed reality paradigm which has the potential to facilitate the rapid capture of implicit inspection knowledge and explicitly represent this in a formalized way. An outline and the results of the development of an early stage prototype - which will form the foundation of a more complex system to address the aforementioned technological challenges identified in the literature survey - will be given

Algorithms for on-line image registration from multiple views

Images of a scene, static or dynamic, are generally acquired at different epochs from different viewpoints. They potentially gather information about the whole scene and its relative motion with respect to the acquisition device. Data from different (in the spatial or temporal domain) visual sources can be fused together to provide a unique consistent representation of the whole scene, even recovering the third dimension, permitting a more complete understanding of the scene content. Moreover, the pose of the acquisition device can be achieved by estimating the relative motion parameters linking different views, thus providing localization information for automatic guidance purposes. Image registration is based on the use of pattern recognition techniques to match among corresponding parts of different views of the acquired scene. Depending on hypotheses or prior information about the sensor model, the motion model and/or the scene model, this information can be used to estimate global or local geometrical mapping functions between different images or different parts of them. These mapping functions contain relative motion parameters between the scene and the sensor(s) and can be used to integrate accordingly informations coming from the different sources to build a wider or even augmented representation of the scene. Accordingly, for their scene reconstruction and pose estimation capabilities, nowadays image registration techniques from multiple views are increasingly stirring up the interest of the scientific and industrial community. Depending on the applicative domain, accuracy, robustness, and computational payload of the algorithms represent important issues to be addressed and generally a trade-off among them has to be reached. Moreover, on-line performance is desirable in order to guarantee the direct interaction of the vision device with human actors or control systems. This thesis follows a general research approach to cope with these issues, almost independently from the scene content, under the constraint of rigid motions. This approach has been motivated by the portability to very different domains as a very desirable property to achieve. A general image registration approach suitable for on-line applications has been devised and assessed through two challenging case studies in different applicative domains. The first case study regards scene reconstruction through on-line mosaicing of optical microscopy cell images acquired with non automated equipment, while moving manually the microscope holder. By registering the images the field of view of the microscope can be widened, preserving the resolution while reconstructing the whole cell culture and permitting the microscopist to interactively explore the cell culture. In the second case study, the registration of terrestrial satellite images acquired by a camera integral with the satellite is utilized to estimate its three-dimensional orientation from visual data, for automatic guidance purposes. Critical aspects of these applications are emphasized and the choices adopted are motivated accordingly. Results are discussed in view of promising future developments

An incremental method for mosaicing of optical microscope imagery

Digital imaging is nowadays widely employed in the field of optical microscopy. One of the most apparent benefits consists in the possibility for the researcher to see the whole biological sample in one image, achieved by collecting all the parts being inspected. Common approaches work in batch mode and rely on known motorized x-y stage offsets of the microscope holder. Or alternatively, the methods are conceived just to provide visually pleasant mosaics off-line, that are often built by altering the photometric values or the geometric properties of the original component images. This work presents an incremental mosaicing method for optical microscopy imagery, compliant with on-line requirements and suitable even for nonmotorized microscopes. The resulting mosaics are very accurate and preserve the consistency of the original images so to be used for further global measurement steps. Nevertheless, the mosaics are visually pleasant so to be used for visual inspection as well. The experimental results obtained in different biological examinations confirm the efficacy of our approach